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Abstract 

We  extend  and  simplify  Smale's  work  on  the  expected  number  of 
pivots  for  a  linear  program  with  many  variables  and  few  constraints. 
Our  method  applies  to  new  versions  of  the  simplex  algorithm  and  new 
random  distributions. 


Random  Linear  Programs  with  Many  Variables 
and  Few  Constraints 

by  Charles  Blair 


1.  Introduction.   In  the  important  papers  [1,2],  Smale  studies 
the  expected  number  of  pivots  required  by  the  simplex  algorithm  for  a 
randomly  generated  problem. 

Smale  considers  a  version  of  the  simplex  algorithm  which  can  be 
viewed  as  a  path-following  procedure.   For  a  fixed  constraint  matrix 
one  begins  with  a  right-hand-side  and  objective  function  for  which 
the  optimal  solution  is  trivial  and  deforms  to  the  actual  ones.   The 
"main  formula"  in  [1,2]  gives  the  expected  number  of  pivots  for  a  fixed 
matrix.   The  "main  formula"  is  then  used  to  prove  the  "main  theorem" — 
that  for  a  fixed  number  of  constraints  the  expected  number  of  steps 
grows  in  a  sublinear  manner  as  the  number  of  variables  increases. 

In  this  note  we  show  how  the  "main  theorem"  can  be  obtained  without 
using  the  geometric  analysis  leading  up  to  the  "main  formula."   This 
simplifies  the  proof.   At  the  same  time  the  result  is  generalized  to 
include  other  versions  of  the  simplex  algorithm  and  other  random  dis- 
tributions of  the  data,  which  are  not  necessarily  amenable  to  geometric 
analysis. 

2.  Statement  of  the  Problem.   We  will  consider  linear  programs  in 
the  form 


max  ex 


Ax  >_  b  (2.1) 

x  >  0  [b  G  Rm,  c  G  Rn] 
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We  will  assume  b  is  a  fixed  non-positive*  vector  and  study  the  ex- 
pected number  of  pivots  when  A  and  c  are  generated  randomly.   The  case 
in  which  b  also  varies  can  be  obtained  as  a  corollary.   The  crucial 
property  assumed  of  the  random  distribution  is 


c  \ 

Let  A'  =  (.)•   For  any  (m+1)  by  n  matrix  B  with  all 

elements  different 

(2.2) 

prob{a!  .  >  a!,  if  b..  >  b.,  for  all  i,j,k}  =  (^r-)11*1 
ij    ik     ij    lk  n! 


In  words  (2.2)  says  that  the  order  of  elements  in  c  and  the  rows 
of  A  are  independent  of  one  another.   This  is  clearly  the  case  if  A 
and  c  are  chosen  using  the  spherical  measure  of  [1],  but  includes 
other  possibilities.   (2.2)  does  imply  that  the  probability  that  any 
two  members  of  the  same  row  of  A  are  equal  is  zero,  and  is  essentially 
equivalent  to  the  symmetry  assumption  in  [2J.   In  section  5  we  show 
how  our  analysis  can  be  extended  to  those  distributions  satisfying 

Let  B  and  B*  be  two  matrices  such  that  each  row  of  one  is 

a  permutation  of  the  corresponding  row  of  the  other.   Then 

(2.2)' 
probU^  >  a»k  iff  b±j  >  b.k;  all  i,j,k}  = 

*       ■k 

prob{a|.  >  a'   iff  b..  >  b.,  ;  all  i,j,k} 

ij    ik      ij    ik 


*To  insure  feasibility.   This  assumption  can  be  avoided  by  minor 
modifications,  which  we  indicate  in  section  5. 
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(2.2)'  would  allow,  for  example,  choice  of  all  elements  of  A  and  c 
from  independent  identically  distributed  discrete  random  variables. 
3.   Idea  of  the  proof.   Consider  the  LP 

max  lOx,  +  5x„  +  9x_  +  4x, 

12     3     4 

-x,  -  2x0  -  7x0  -  10x.  >  -5  (3.1) 

1      2      3       4  — 

-7x_  -  8x.  +  3x„  +  x.    >  -15 
1      1  3     4   — 

V  V  V  X4  -° 

We  can  see  immediately  that  the  optimal  solution  to  (3.1)  must 
have  x  =  x,  =  0.   This  is  because  the  column  of  A'  (10,-1,-7) 
(5,-2,-8)  and  (9,-7,3)^(4,-10,1).   In  general  a  column  of  A'  is 
said  to  be  undominated  if  there  is  no  other  column  at  least  as  large 
in  every  row.*   We  are  concerned  with  those  versions  of  the  simplex 
algorithm  which  satisfy 


No  variable  corresponding  to  a  dominated  column  of  A' 

(3.2) 
enters  the  basis  at  any  iteration. 


The  path-following  algorithm  in  [1,2]  satisfies  (3.2).   Other  ver- 
sions of  the  simplex  algorithm  satisfying  (3.2)  include 

If  possible,  choose  a  surplus  variable  as  entering 
variable.   Otherwise  choose  the  entering  variable  with      (3.3a) 
largest  reduced  cost. 


*If  we  allow  the  possibility  that  several  columns  of  A'  are  equal, 
we  say  the  leftmost  equal  column  dominates  the  others  but  not  vice 
versa. 
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Delete  all  dominated  columns  from  Che  tableau  at 
the  beginning,  then  use  any  version  of  the  simplex  (3.3b) 

algorithm.   (See  [3,4]  for  algorithms  for  identifying 
the  set  of  dominated  columns.) 

The  largest  reduced  cost  rule  for  choosing  entering  variables  does 

not  satisfy  (3.2).   Consider  the  program:* 

max     5Xn  +  3X„  +  X0  +  X, 
1      2     3     4 

subject        -X,    +   X„   -    5X0   -    3X.    >  -10 
J  12  3  4  — 

to 

-3X.    -  X     -  X,  -  X.     ►  -40 

1  2  3  4  — 

+2X,    -   X     +   2.9X„  +   3X.    >   -10        12.5        2.5 
12  3  4  — 

0        31   2/3      8   1/3      0 

The  column  for  X.  dominates  the  column  for  X-  but  X_  enters  the  basis 
4  3      3 

at  the  third  iteration  (see  table  at  right). 

If  the  matrix  A'  has  U  undominated  columns  then  any  version  of  the 
simplex  algorithm  which  satisfies  (3.2)  will  use  at  most 

(    )  <  (U+m)   pivot  steps  to  find  the  optimum  (or  discover  the  LP  is 

ra   — 

unbounded) . 

To  establish  bounds  on  the  expected  number  of  pivots  it  suffices 
to  show  that  the  size  of  U  grows  slowly  with  n,  for  m  fixed. 

Theorem  3.1:   Fix  m.   There  are  constants  C  <  e   and  C„  such 


xl 

X2 

X3 

X4 

0 

0 

0 

0 

10 

0 

0 

0 

12.5 

2. 

5 

0 

0 

that,  for  n  sufficiently  large  and  A'  generated  so  that  (2.2)  holds 
the  probability  that  there  are  at  most 


*This  example  also  shows  that  the  "maximum  increase"  rule  for 
choosing  entering  variables  does  not  satisfy  (3.2).   This  corrects  an 
erroneous  claim  in  the  original  manuscript. 


C2(Ln  B)(-«)L«C-H)  +  1  (3.4) 

undorainated  columns  is  at  least 

,     .  (nri-l)Ln(nri-l)   Ln  n                          .         . 

1  -  (Ln  n)            C  (3.5) 

We  prove  this  result  in  section  4.   From  this  we  easily  obtain 

Theorem  3.2:   Fix  m.   Let  P(n)  be  the  expected  number  of  pivots 
for  an  A'  generated  by  (2.2).   Then 


m(ro+l)Ln(m+l)  +  m 


jn 


Lim  P(n)/(Ln  n)  v   '   v   '  _<  C£  (36) 

n>°° 


Proof:   With  probability  )■  (3.5)  the  number  of  pivots  will  be 
(U+m)  ,  where  U  is  given  by  (3.4).   In  the  remaining  cases,  the 

worst  number  of  pivots  is  n  .   The  bound  on  C.  implies  that 

,  .   /t    \  k„Ln  n  m    .   .       , 
Lim  (Ln  n)  C.    n  =  0,  for  any  k. 

n  QED 

4.   The  Number  of  Undominated  Columns  in  a  Randomly  Generated  Matrix. 

The  assumption  (2.2)  has  the  effect  of  converting  our  problem  to 
the  following: 

Suppose  we  generate  an  m  by  n  matrix  A  in  which  each  row  is  a  per- 
mutation of  {1,  2,  ...  n}  chosen  independently  from  the  uniform 
distribution.   Study  the  behavior  of  the  random  variable  U  =  number  of 
undorainated  columns. 

This  problem  has  been  studied  in  [5,6],  and  we  suspect  there  are 
earlier  references.   [5,6]  show  that  the  expected  value  of  U  is 
0(log  n)     for  n  large.   However,  we  need  different  information  to 
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obtain  Theorem  3.2.   The  admittedly  crude  approach  in  this  section 
uses  Stirling's  formula  and  elementary  probability. 

Lemma  4.1:   Suppose  we  choose  sets  of  size  an,  3n  independently 
from  a  set  of  n  elements,  0  <_  a,  3  <  ~.   There  is  E  <  1  such  that 
the  probability  that  the  intersection  has  exactly  —  a3n  elements  is 

<_   Ea6n. 

Proof .   The  probability  in  question  is 

an.'gnla'b.'B'n!  K         J 


n.^aSn!S(l-  -|a)n!o(l-  -|e)n!(l-g-a+  -|aB)n! 


where  a  +  a'  =  3  +  3f  =  1.   If  we  apply  Stirling's  formula  in  the  form 

~x  1  1  u.  l  1 

""  2  -n,„  ,2  y      ,  /    2  -ti/0  ,2  l/12n    .   .      .     T  „-   .  .  „ 
n    e   (2tt)  _<  n!  _<n    e   (2ir)  e      and  the  estimate  Ln(l+X)  _<  X, 

■|a3n 

X  <  1,  we  obtain  the  upper  bound  R0(2/e)     ,  where  0  = 

exp(12n~1(a~1+B~1+a'~1+3'~1))  and  R  <_  SivnaQ)'1 .      For  nag  sufficiently 

1/2 
large  (_>_  10)  we  may  take  E  =  (2/e) 

QED 
Lemma  A.  2:   There  is  D  <  1  and  Q  such  that,  if  a(3n  >  Q  the  proba- 


bility that  the  intersection  as  in  lemma  4.1  has  <_—  ctSn  elements  is 
<  Da6n. 


Proof:  If  we  denote  by  P   the  probability  that  the  intersection 

1  K, 

has  exactly  k  elements  and  look  at  the  formula  for  P   in  factorials, 

we  see  that  for  agn  sufficiently  large  and  k_<_y  a3n,  P.  i  _i.~2"Pk* 
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Hence  the  probability  is  bounded  by  Z2  ~^E    =  2E    .   If  a3n  is  suf- 
ficiently large  D  can  be  chosen  suitably. 

OED 

We  have  abbreviated  the  proofs  of  lemmas  4.1  and  4.2  partly 
because  the  details  are  cumbersome,  and  partly  because  these  are  known 
properties  of  the  hypergeometric  distribution. 

Lemma  4.3:   There  is  a  B  <  1  and  Q  such  that  if  a  <  —  a  n  >_  0 

and  A  is  chosen  so  that  (2.2)  is  satisfied,  then  the  probability  that 

m 
A  has  a  column  in  which  each  entry  is  >  (l-a)n  is  at  least  1-B 


Proof:   The  sets  consisting  of  those  columns  whose  first 
(second)  entry  is  among  the  an  largest  in  the  first  (second)  row  are 

both  of  size  an  and  chosen  independently.   Hence  the  probability  that 

1   2 
there  are  at  least  —an  columns  whose  first  two  entries  are 

2 
a  n 
(l-a)n  is  _>_  1  ~  D    ,  by  lemma  4.2.   Next  we  consider  the  set  of 

columns  whose  third  entry  is  among  the  an  largest  in  the  third  row. 

This  set  is  independent  of  the  set  of  columns  whose  first  two  entries 

1   2 
are  among  the  an  largest.   We  may  apply  lemma  4.2  with  3  =  -r-  a   to 

1   3 
conclude  that  the  probabilitv  that  there  are  at  least  -ran  columns 

v  J  4 

2      -j<x   n 
with  entries  )     (l-a)n  is  _>_  (1-D    )(1-D~    ).   Continuing  in  this  way 

we  conclude  that  the  probability  that  there  are  at  least  2    a'  n 

columns,  all  of  whose  entries  are  among  the  an  largest  is  at  least 

,      1  3         -k+2  k  -k+2  k 

2      ^ra  n.    > .  _2    a  nN  .  .    ,  _2    an     . ,    _ . 
(l_Da  n)(l_D2    )...(1~D        )  2.  1  "  kD  (4.3) 

3  and  0  may  be  chosen  so  that  the  conclusion  follows. 

QED 


If  the  matrix  A  has  at  least  one  column  with  all  entries  among  the 
an  largest,  then  that  column  will  dominate  all  those  columns  whose 
entries  are  all  _<_  (l-a)n.   This  would  imply  that  the  number  of  undomi- 
nated  columns  is  at  most  man.   We  may  repeat  this  analysis,  con- 
centrating on  those  columns  of  A  corresponding  to  the  an  largest 
entries  in  row  j,  1  _<_  j  l_  m.   This  idea  is  carried  out  formally  below. 

Definition  4.4:   Let  A  and  a, ,  ...  a  <„ — be  fixed.   For 
1'      s  — 2 


j  .  £  {1,  2,  ...  m}  1  <  i  _<_  s  define  a  subset  of  the  columns  of 

A,  T(j  ,  ...,  j  )  inductively  as  follows:   (1)  T(j.)  is  the  columns  of 

A  which  have  the  an  largest  entries  in  the  j.  component.   (ii) 

T(j,,  •••  j    )  is  a  subset  of  T(j  ,  ...  j  )  which  has  the 

ana^  ...  a  ,,n  largest  entries  in  the  j  , n  component. 
1  2       q+1      &  Jq+1     v 

Lemma  4.5:   If  A  is  randomly  generated  and  satisfies  (2.2),  the 

probability  that  A  has  _<_  m  a,a9  ...  an  undominated  columns  is  at 

least 

L     ,  oc,a„. .  .a  ,  a  n 
1  -  E   m^B  1   2    I""1  q  .  (4.4) 

q-1 

m 

Proof:   Let  P(j,,  ...  j  )  be  the  probability  that  n  T(j, ,  ...  j  ,  i) 
l       q  i-1  q 

is  non-empty.   By  lemma  4.3,  the  probability  of  this  event  is  at  least 

k 
a, a«. . .a  a  , , n 

1  -  B  1   Z  q  q+1  .  (4.5) 


If  this  event  occurs,  it  implies  that  every  member  of  T(j.,  . . . j  )  - 

m 

U  T(i,,  ...  i  ,  i)  is  dominated  by  a  member  of  the  intersection.   If 

.  ,    1       q 
i=l 
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the  events  corresponding  to  P(j,,  ...  j  )  occur  for  all  0  _<_  q  <_  L  -1 

and  all  sequences*  i,,  ...  i   all  occur  then  the  dominated  columns 
M        Jl      Jq 

must  all  be  in  T(in,  ...  j_ ) ,  where  the  union  is  taken  over  all 

1  Ij 

sequences  of  length  L.   The  number  of  elements  in  the  union  and  the 

lower  bound  on  the  probability  of  all  the  events  occuring  are  as 

stated  (note  we  do  not  assume  the  events  are  independent). 

OED 

Theorem  4.6:   For  any  0,  if  A  is  randomly  generated  and  satisfies 
(2.2)  the  probability  that  A  has  less  than 

,. ,,    .  mLn  m+1  ,,  .  N 

0(Ln  n)       e  (4.6) 

undominated  columns  is  at  least 

.    f  .mLn  mOLn  n  ,,  7, 

1  -  (Ln  n)      B  (4.7) 

for  n  sufficiently  large. 

Proof:   We  apply  lemma  4.5.   Let  L  =  m  Ln  Ln  n  and  a  =  (QLn  n/n) 


m      m     „,     L  ,,    vmLn  m,.,    x/  ,„    xv 

i.  =  a.  a.,.  , 
l    l  l+l 


Let  a_.  =  a.a.,,.   Then  m'a,  ...an  =  (Ln  n)     (QLn  n)(n/QLn  n)  , 


where  v  =  (1-1/m)   '.    exp  (-L/m)  =  1/Ln  n.   Hence  n  [   e.   To  justify 

(4.7)  note  that  the  expression  (4.4)  is  in  this  case 

QLn  n    q-1         QLn  n,  L. 
1  -  B      (ZnP   )  >  1  -  B      (m  ). 

—  QED 


n 
*The  event  corresponding  to  q  =  0  is  nT(i)  non-empty. 

1 
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To  obtain  Theorem  3.1  from  4.6  replace  ia  by  n  +  1  and  choose  Q  so 
tnat  B   <  e 


5.   Concluding  Remarks. 

The  expression  (3.6)  compares  unfavorably  with  the  expected  value 
obtained  in  [1],  which  is  essentially  (Log  n)     .   I  conjecture  that 
the  analysis  in  section  4  can  be  improved  to  obtain  this  stronger 
result.   It  should  also  be  mentioned  that  [1,2]  also  use  the  dominance 
idea  (see  the  definition  of  the  X-sets  in  section  5  of  [1]).   The 
contribution  of  this  paper  is  to  show  that  the  key  properties  for  this 
problem  are  (2.2),  (3.2),  and  dominance,  rather  than  geometric 
analysis . 

These  results  extend  to  cases  in  which  b  is  not  necessarily 
feasible.   To  determine  feasibility,  one  adds  m  columns  corresponding 
to  artificial  variables.   Since  property  (3.2)  still  holds,  the  exten- 
sion is  immediate. 

Similarly,  the  possibility  of  a  degenerate  problem  does  not  affect 
the  analysis.   The  standard  device  of  perturbing  the  right-hand  side 
(lexicographic  pivot  rules)  may  be  implemented  in  such  a  way  that 
(3.2)  still  holds. 

To  replace  (2.2)  by  (2.2)'  we  must  show  that  the  analysis  in  sec- 
tion 4  still  works.  For  any  matrix  B  let  A  be  a  matrix  with  all  ele- 
ments different  such  that  a..  <  a.,  if  b..  >  b.,  .   The  number  of  undo- 

ij     lk     ij     lk 

minated  columns  of  B  is  <  the  number  of  undominated  columns  of  A. 
Further,  if  we  permute  row  elements  of  3  and  A  to  obtain  B* ,  A*  #  un- 
dominated B*  \  :.-"  undominated  A*.   Thus  the  average  number  of  undomi- 
nated columns  of  B  as  we  go  through  all  possible  permutations  is 
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smaller  than  average  number  for  A,  to  which  the  analysis  in  section  4 
applies. 

The  results  in  this  paper  depend  in  a  crucial  way  on  n  >>  m.   It 
might  be  useful  to  look  for  dominance  every  few  iterations,  since  new 
dominant  columns  may  appear.   Also,  it  might  help  to  extend  the 
concept  of  dominance  to  cases  in  which  a  non-negative  multiple  of  one 
column  dominates  another. 

Finally,  we  wish  to  mention  work  of  Megiddo  [7]  on  specialized 
algorithms  for  LPs  with  many  variables  and  few  constraints. 
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